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Credits 

Work  performed  at  the  NPS  Vision  Lab  and  MOVES  -JJJ 
Institute 

in  collaboration  with  the 


Remote  Sensing  Center  at  NPS 
Unmanned  Systems  Lab  at  NPS 

Student  collaborators: 

Rich  Morrison 
Justin  Jones 
Rob  Zaborowski 

Faculty  collaborators: 

Simson  Garfinkel 
Chris  Olsen 
Amela  Sadagic 
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Example:  detect  US  Marines  and  their  posture 


methods  to  determine  parts: 

•  manual  designation 

•  random  patches,  clustered 

•  random  patches,  learned 


Parts'  Relationships 

Learn  from  training  data;  statistics  determine  the  distribution 
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Probabilistic  Part  Detections 


Feature  Thresholded  Combined  classifier 

output  output 


MOVES 


Probabilistic  Part  Detections 

Part  probability  Thresholded  Combined  classifier 


output 


Second  part  detector 
Produces  a  different  set  of 
detections. 
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Probabilistic  Part  Detections 

Feature  Thresholded  Combined  classifier 

output  output 


Probabilistic  Part  Detections 
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Combined 
classifier 
at  iteration  1 0 


Probabilistic  Part  Detections 


Part 
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output 
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Adding 
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Combined 
classifier 
of  200  parts 
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Final 

classification 


Posture  Recognition 


for  BASE-IT 


http://www.movesinstitute.org/base-it/ 
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Juan  P.  Wachs,  Deborah  Goshorn,  Noah  Lloyd-Edelman 
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Hierarchical  Learning 

c 

m 

•  primate  vision 

•  computer  vision  in  the  70s  (structural  models) 

•  rediscovered  now: 

-  more  powerful  learning  methods, 

-  better  local  parts  ("features"),  more  invariant 

-  faster  computers 
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At  the  NPS  Vision  Lab: 
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development  of: 

methods  for  automated  object  recognition  in 
image  and  video  data,  of  untypical  objects,  as 
opposed  to  face  or  pedestrian  detection 
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Application  to  IED  Prevention 
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Where  is  Waldo?* 
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Tools  exist  for: 

•  determination  of  approximate 
disk  contents 

•  restoration  of  deleted  files 

•  text  analysis  of  emails,  temporary 
internet  files,  etc 

Lack  of  tools  for: 

•  video  and  still  image  content 
analysis 

*)  Waldo,  or:  the  wiring  diagram, 
the  photo  of  the  chemical 
compound,  the  weapon, ... 
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al  qaeda,  ‘^'6^-^0^0,200,000  text  hits 

al  qaeda  recruiting,  ^l^P-^'64' 578,000  text  hits,  38  videos 

jihad  training  camp,  254,000  text  hits,  38  videos 

jihad  recruiting,  311,000  text  hits,  9  videos 
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translations  via 
TranStar  translator 
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Training  an  AK  47  Detector 


Thesis  work  of  Justin  Jones 


Positive  Image 


1146  Positive  Images 
5668  Negative  images 

Normalized,  Grey  Scale 

—  20x40  for  whole  AK 

—  20x20  for  Left  and  Right  Half. 

Negative  Image 
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Structural  Classification  with  a  :q 

Support  Vector  Machine  m 

•  Left  and  Right  Detectors  were  run  over  the  training  set. 

•  Detections  in  the  annotated  box  are  considered  true  detections. 

•  Detections  outside  the  annotated  box  are  false  detections. 

•  Vector  is  Difference  between  Left  and  Right  CenterX's,  Y's,  and  Radii,  all 
normalized  by  the  mean  radius  of  the  2  detections. 


Left  Detector  Right  Detector 


Performance 
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Parts-Based  Object  Detection 
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Benefits: 

•  smaller  areas,  speed,  special  hardware 

•  permits  variability  instead  of  a  rigid  whole  object 
(wheels,  for  varying  car  wheelbase) 

•  reuse  parts  for  multiple  objects 
(wheels  for  cars,  motorcycles,  even  clocks) 

Basic  research  questions: 

•  what  are  ideal  parts? 

•  how  are  they  best  combined? 
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Questions? 

Contact: 

Mathias  Kolsch 
kolsch@nps.edu 


NPS  Vision  Lab 


Your  are  currently  viewing:  Home 

NPS  Vision  Lab 

The  NPS  Vision  Lab  is  a  research  and  education  lab  in  the  MOVES  Institute  and  the 
Computer  Science  Department  at  the  Naval  Postgraduate  School  in  Monterey,  CA.  Our 
expertise  is  at  the  crossroads  of  computer  vision,  computer  graphics  and  human- 
computer  interaction.  We  collaborate  with  NPS-wide  efforts  on  training  systems, 
robotics  and  autonomous  systems,  sensor  networks  and  embedded  systems.  We 
strive  to  accomplish  projects  with  educational  goals  while  incorporating  and 
advancing  current  research  into  prototype  systems. 
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News  updates 

MOVES  Director  CDR  Joe  Sullivan  gave  an  interview  for  the  E 


ig,  talking  about  some  of  the  technologies  developed  here  in  the  NPS 


TheVelodyne  LIDAR  that  was  purchased  by  the  NPS  Physics  Dept,  and  the  MOVES  Institute  in  a  joint  effort  has  enjoyed  plenty  of  attention.  We  mounted  it  on 
a  truck  and  drove  it  through  urban  areas.  Here  are  a  few  images: 


Lidar  scan  of  an  urban  scene 


http://vision.movesinstitute.org/ 


Backup  Slides 


z 


m 


MOVES 


Technical  Challenges 
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•  accurate  overlay  of  geospatial  information 

—  GPS  location  information  is  not  accurate  enough  in  city 
streets  and  might  even  be  jammed. 

•  acquire  dynamic  3D  model  of  the  environment 

—  location  estimation  of  the  own  combat  vehicle  within  a 
street 

•  extract  sufficient  building  geometry 

—  to  place  3D  annotations  accurately  on  windows,  doors, 
street  corners  etc. 

—  Simultaneous  Localization  And  Mapping  (SLAM) 

•  suitable  visualizations  (icons,  overlays,  etc) 

•  field-feasible  geo-registered  input 
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Velodyne  HDL-64E 
+  Point  Grey  Ladybug 


Lidar  Data  -  X3D  Rendering 
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Calibration  Research 
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GPS  resolution:  1000mm  or...? 
LiDAR  resolution:  2-1 0mm 

what  about  accuracy? 
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Finding  the  Markers 


VLS  Data  Processing 


LIDAR  Sensor 
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Packet  Capture  file  (.pcap) 
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LIDAR  Input 
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Convert  to 

Cartesian  Point 


Fill  PointBuffer 


Segmentation 
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Arc  Finder 
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Axis  Line  Finder 


Cylinder  Finder 


Cone  Finder 
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Fiducial  Marker 
_ Finder _ 


Packet  Processing 


J 


Options 


calibration  (.xml) 


SegmentlndexBuffer 


X 


CircleArcBuffer 
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AxisLineBuffer 
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CylinderBuffer 
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ConeBuffer 


Pose  Estimate 


Geometric  Classes: 

•  Cartesian  Point 

•  Segment 
•Arc 

•  Line 

•  Cylinder 
•Cone 

•  Sphere 

Primitive  Buffer: 

•  Point  Buffer 

•  Segment  Index  Buffer 

•  Circle  Buffer 

•  Consensus  Set  Buffer 

•  Axis  Line  Buffer 

•  Cylinder  Buffer 

•  Cone  Buffer 


Point  Labels: 

•  Constant  gradient 

•  Gradient  discontinuity 

•  Constant  range 

•  Range  discontinuity 

•  Start  of  a  new  segment 


render  Results 


recordResults 
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Marker  Position  Estimation,  Cylinder 
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Marker  Position  Estimation,  Cone 
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Contributions 
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Very  accurate  ground-truth  (orders  of  magnitude 
better  than  GPS) 

Real-time  panoramic  video  and  depth  fusion 
Vehicle-based  Augmented  Reality 
system  prototype 
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